Fitting surfaces to data with covariances

We are concerned with solving an equation whose form is applicable to a wide class of problems arising in computer vision. The equation typically relates image point locations to the parameters of some appropriate model. We assume that each measured datum is accompanied by a covariance matrix that characterises the uncertainty of the measurement. Noisy data are assumed to be in plentiful supply, implying that our problem is overdetermined. To tackle noise, the problem is transformed to one of least squares minimisation. In this sense, we are concerned with fitting a surface to data and their covariances. Estimation of the fundamental matrix and conic fitting are two vision problems subsumed by our general form. The paper has two principal concerns: the establishing of a suitable cost function for our general problem, and the deriving of an effective scheme for minimising the cost function. A weighted least squares cost function is obtained by considering a maximum likelihood formulation, and applying certain necessary approximations that render the problem tractable. A fundamental Newton-like iterative scheme is then generated for determining a minimiser of the cost function. Unlike alternative approaches, such as Sampson’s method or the renormalisation technique, the new scheme has as its theoretical limit the minimiser of the cost function. Various minimisation schemes are tested on the problem of estimating the fundamental matrix from corresponding points and their covariances. When the data exhibit noise that is anisotropic and inhomogeneous, those methods that make use of covariance matrices perform markedly better than more traditional methods that do not. None of the methods outperforms the fundamental scheme. In the event that genuine covariance matrices are unavailable, the fundamental scheme may instead be used with default identity covariances matrices. In this mode, the fundamental scheme significantly outperforms the ordinary least squares and Sampson approaches. The fundamental numerical scheme thus has the advantages of being unsurpassed in testing as a general technique, simply expressed, efficient, and a theoretically sound means of determining the cost function minimiser.